Papers with BERT-based language models
Understanding Pre-trained BERT for Aspect-based Sentiment Analysis (2020.coling-main)
Copied to clipboard
| Challenge: | Recent studies show impressive results on aspects-based sentiment analysis tasks. |
| Approach: | They analyze the attentions and learned representations of BERT for aspects-based sentiment analysis tasks. |
| Outcome: | The proposed model can be used for aspects-based sentiment analysis (ABSA) but it is not clear how it can provide important features for downstream tasks. |
Data Selection for Fine-tuning Large Language Models Using Transferred Shapley Values (2023.acl-srw)
Copied to clipboard
| Challenge: | Large language models (LMs) have been shown to be highly effective for identifying harmful training instances, but dataset size and model complexity constraints limit the ability to apply Shapley-based data valuation to fine-tuning large pre-trained language models. |
| Approach: | They propose an algorithm that aggregates Shapley values from subsets for valuation of entire training set and a value transfer method that leverages value information extracted from a simple classifier trained using representations from the target language model. |
| Outcome: | The proposed method outperforms existing methods on benchmark datasets and can filter fine-tuning data to increase language model performance compared to training with the full fine-uning dataset. |
MDCSpell: A Multi-task Detector-Corrector Framework for Chinese Spelling Correction (2022.findings-acl)
Copied to clipboard
| Challenge: | Chinese Spelling Correction (CSC) is a task to detect and correct misspelled characters in Chinese texts. |
| Approach: | They propose a general detector-corrector multi-task framework which exploits the visual and phonological features of the misspelled characters and minimizes their misleading impact on the context. |
| Outcome: | The proposed framework outperforms the state-of-the-art methods on Chinese Spelling Correction tasks. |
Noise, Novels, Numbers. A Framework for Detecting and Categorizing Noise in Danish and Norwegian Literature (2024.emnlp-main)
Copied to clipboard
Ali Al-Laith, Daniel Hershcovich, Jens Bjerring-Hansen, Jakob Parby, Alexander Conroy, Timothy Tangherlini
| Challenge: | This study examines the literary perceptions of noise during the Scandinavian "Modern Breakthrough" period (1870-1899). |
| Approach: | They propose a framework for detecting and categorizing noise in literary texts from the late 19th century. |
| Outcome: | The proposed framework can be applied to Danish and Norwegian literature from the late 19th century. |
Syntax Matters: Towards Spoken Language Understanding via Syntax-Aware Attention (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on SLU systems have focused on integrating syntactic information into language models. |
| Approach: | They propose a model where attention scopes are constrained based on syntactic relationships. |
| Outcome: | The proposed model improves on three datasets and can be integrated into other language models to further boost their performance. |